Discussion ¶
We have performed various types of analysis on the characters in the Harry Potter universe including mapping the characters as a social network and analyzing their descriptibe wikipedia texts. We started out with a hypothesis that the division into the 'Hogwarts' houses would be a central theme in the data. However, it was quickly apparent from the distribution of characters into houses that the majority of the data points had 'Unknown' as the house attribute. When we compared the community division based on houses to the algorithmic Louvain division, it was also clear that the house division did not fit the data well. We could tell this from comparing the modularity. This meant our initial hypothesis did not hold. This is likely because the data contains many more characters than just the ones in the Harry Potter books. There are three movies, the 'Fantastic beasts' series, with many characters from America and the rest of the world that are contained in the 'Unknown' house. Knowing this it is quite understandable that the school houses are too simple of a division. Furthermore, in our analysis of mixing patterns we found that nodes of the same houses do not tend to reference each other more than a random network would. Though this is surprising, it further explains why house community division was far worse than the Louvain division.
We did still analyze the texts from the characters in the 'Hogwarts' houses to see if some of our initial thoughts held up. We expected to see different words in focus for the four houses. However, many of the words were the same, like 'Harry' and 'Hogwarts'. There were some differences in the words with lower TF-IDF scores, but generally the house wordclouds did not show significant differences in house themes. The biggest standout was the word 'Newt' in the 'Hufflepuff' wordcloud. As 'Newt Scamander' is the main character of the 'Fantastic beasts' series his importance makes a lot of sense.
Creating individual wordclouds showed interesting results. There were clearer differences in the words, and they matched with our expectations for the characters. The words with highest TF-IDF scores were names, which is understandable as names are pretty unique and often not used in many contexts. Hereafter the wordclouds mostly showed nouns and other factual words. This makes sense for a wikipedia page. However, since the wiki pages are written by fans we could not have simply expected factual, formal language and an analysis like the one we made is usefull to understand the trustworthiness of the pages. They seems to thouroughly describe the events in the lore, without presenting subjective perspectives. However, an add-on for the project could be to make a semantic sentiment analysis, which we would expect would come out neutral.
Diving back into our network analysis, we clearly showed that the Harry Potter network differentiates itself from a random network. Just looking at the in- and out-degrees it is clear that some characters are very central while others are without much importance. This distribution was very different in the random network. The clustering coefficient supported this discovery. With a clustering coefficient of 0.3 for the Harry Potter network and 0.003 for the random network, the Harry Potter universe has many more clusters than possible if the characters were randomly connected. This high difference was likely based on the 200 nodes in small fully connected clusters. A add-on to the investigation could have been to only look at the largest connected component. From our assortativity measures, we did not see exactly what we expected. Nodes with some in-degree did not seem particularly connected to other nodes with the same in- or out-degree, but the assortativity was more significantly different from random for the out-degree of the source nodes. The measures looking at the in-degree of the source node were still about twice as large as the same measures for the random network, however from the numbers alone it is hard to tell if this is significant. This is one of the main limitations of our network analysis. Though, we have compared all results to a random network, a better analysis would look at distributions of random networks. By computing statistics on 100+ random networks, we could have found the 95% confidence intervals for the different metrics and concluded if the Harry Potter network metrics were statistically different from one. This could be an interesting add-on for a future project. Furthermore, we could have made a randomization experiment to investigate if the modularity measure for both the house division and the louvain division are different from a random graph. This could be done with double edge swap and would also be and interesting add-on.
We computed the average shortest path of the Harry Potter network and the random network and again found different values, with the Harry Potter network having the shortest paths. This analysis has some major issues. Firstly, the largest connected component of the random network is three times as big as the Harry Potter network. This means that the maximum possible shortest path will be much larger in the random network than in the Harry Potter network. The second issue with this metric is that it considers the weights of the edges in a way were a high weight is considered a longer path. However, with the way we modelled our network a high weight means that the characters are strongly connected. A better way to consider shortest path, would have been to evaluate the negative of the weights or inverse the weights. This issue moves into the closeness centrality metric, where we found that the random network contains more nodes with a high closeness centrality score than the Harry Potter network. It does seem possible since the random network has lower average degrees, and is therefore likely better connected across the whole graph. However, due to the issues decribed above, we cannot trust these metrics.
